System and method for classification determination

ABSTRACT

The present disclosure provides a system and method for classification determination of a structure. The method may include obtaining image data representing a structure of a subject. The method may also include determining a plurality of candidate classifications of the structure and their respective probabilities by inputting the image data into a classification model. The classification model may include a backbone network for determining a backbone feature of the structure, a segmentation network for determining a segmentation feature of the structure, and a density classification network for determining a density feature of the structure. The method may further include determining a target classification of the structure based on at least a part of the probabilities of the plurality of candidate classifications.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Chinese Application No.201811630064.4, filed on Dec. 28, 2018, the contents of which are herebyincorporated by reference.

TECHNICAL FIELD

The present disclosure generally relates to systems and methods forimaging, and more particularly, to systems and methods for determining atarget classification of a structure in an image.

BACKGROUND

Medical imaging for diagnosis and treatment may be implemented bysystems including, e.g., a X-ray imaging system, a positron emissiontomography (PET) system, a magnetic resonance (MR) system, a computedtomography (CT) system, a single-photon emission computed tomography(SPECT) system, a radioisotope imaging system, etc. For instance, CT isa widely used diagnosis technique. However, image analysis by ahealthcare provider, e.g., a doctor, may be time consuming, and/orintroduce inconsistencies or errors caused by differences betweenhealthcare providers. In recent years, computer-aided diagnosis (CAD)has been developed to reduce work intensities of doctors as well asimprove the disgnostic accuracy. However, structures, such as pulmonarynodules and pleural nodules of small sizes, are still difficult todistinguish their classfications (e.g., malignancies). Thus, it isdesirable to develop a system and method for automated determination ofa target classification of a structure more accurately and efficiently.

SUMMARY

In a first aspect of the present disclosure, a system is provided. Thesystem may include at least one storage medium including a set ofinstructions, and at least one processor in communication with the atleast one storage medium. When executing the set of instructions, the atleast one processor may be configured to direct the system to performoperations including obtaining image data representing a structure of asubject, determining a plurality of candidate classifications of thestructure and their respective probabilities by inputting the image datainto a classification model, wherein the classification model includes abackbone network for determining a backbone feature of the structure, asegmentation network for determining a segmentation feature of thestructure, and a density classification network for determining adensity feature of the structure, and determining a targetclassification of the structure based on the probabilities of theplurality of candidate classifications.

In some embodiments, the backbone network may include a plurality ofdown-sampling layers, each down-sampling layer including a convolutionlayer, a batch normalization layer, and a rectified linear unit layer.

In some embodiments, the determining a plurality of candidateclassifications of the structure by inputting the image data into aclassification model may include obtaining the backbone feature, thesegmentation feature, and the density feature by inputting the imagedata into the backbone network, the segmentation network, and thedensity classification network, respectively, and determining aprobability of each of the plurality of candidate classifications of thestructure based on the backbone feature, the segmentation feature, andthe density feature.

In some embodiments, the determining the probability of each of theplurality of classifications of the structure based on the backbonefeature, the segmentation feature, and the density feature may includedetermining an identification feature of the image data by combining thebackbone feature, the segmentation feature, and the density feature, anddetermining the probability of each of the plurality of candidateclassifications of the structure based on the determined identificationfeature of the image data.

In some embodiments, the determining an identification feature of theimage data by combining the backbone feature, the segmentation feature,and the density feature may include converting the backbone feature, thesegmentation feature, and the density feature into a one-dimensionalbackbone feature vector, a one-dimensional segmentation feature vector,and a one-dimensional density feature vector, respectively, determininga one-dimensional identification feature vector by splicing theone-dimensional backbone feature vector, the one-dimensionalsegmentation feature vector, and the one-dimensional density featurevector, and designating the one-dimensional identification featurevector as the identification feature of the image data.

In some embodiments, the classification model may be trained accordingto a focal loss function, at least one weight of the focal loss functioneach of which corresponds to one of the plurality of candidateclassifications being different from weights of the focal loss functioncorresponding to the remainder of the plurality of candidateclassifications.

In some embodiments, the obtaining image data representing a structureof a subject may include obtaining original image data including arepresentation of the structure of the subject, and determining theimage data by preprocessing the original image data.

In some embodiments, the determining the image data by preprocessing theoriginal image data may include generating a resampled image byresampling the original image data according to a resampling resolution,segmenting the resampled image into image crops according to a center ofthe structure, and determining the image data by normalizing the imagecrops according to a normalizing function.

In a second aspect of the present disclosure, a system is provided. Thesystem may include at least one storage medium including a set ofinstructions, and at least one processor in communication with the atleast one storage medium. When executing the set of instructions, the atleast one processor may be configured to direct the system to performoperations including obtaining a preliminary classification model, andgenerating a classification model for determining a plurality ofcandidate classifications of a structure of a subject represented inimage data by training the preliminary classification model using afocal loss function, wherein at least one weight of the focal lossfunction each of which corresponds to one of the plurality of candidateclassifications being different from weights of the focal loss functioncorresponding to the remainder of the plurality of candidateclassifications.

In a third aspect of the present disclosure, a method is provided. Themethod may be implemented on a computing device having at least oneprocessor and at least one computer-readable storage device. The methodmay include obtaining, by the computing device, image data representinga structure of a subject, determining, by the computing device, aplurality of candidate classifications of the structure and theirrespective probabilities by inputting the image data into aclassification model, wherein the classification model includes abackbone network for determining a backbone feature of the structure, asegmentation network for determining a segmentation feature of thestructure, and a density classification network for determining adensity feature of the structure, and determining, by the computingdevice, a target classification of the structure based on theprobabilities of the plurality of candidate classifications.

Additional features will be set forth in part in the description whichfollows, and in part will become apparent to those skilled in the artupon examination of the following and the accompanying drawings or maybe learned by production or operation of the examples. The features ofthe present disclosure may be realized and attained by practice or useof various aspects of the methodologies, instrumentalities andcombinations set forth in the detailed examples discussed below.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is further described in terms of exemplaryembodiments. These exemplary embodiments are described in detail withreference to the drawings. The drawings are not to scale. Theseembodiments are non-limiting exemplary embodiments, in which likereference numerals represent similar structures throughout the severalviews of the drawings, and wherein:

FIG. 1 is a schematic diagram illustrating an exemplary imaging systemaccording to some embodiments of the present disclosure;

FIG. 2 is a schematic diagram illustrating hardware and/or softwarecomponents of an exemplary computing device according to someembodiments of the present disclosure;

FIG. 3 is a schematic diagram illustrating hardware and/or softwarecomponents of an exemplary mobile device according to some embodimentsof the present disclosure;

FIG. 4 is a block diagram illustrating an exemplary processing deviceaccording to some embodiments of the present disclosure;

FIG. 5 includes a flowchart illustrating an exemplary process fordetermining a target classification of a structure of a subject in animage according to some embodiments of the present disclosure;

FIG. 6 includes a flowchart illustrating an exemplary process fordetermining a target classification of a structure of a subject in animage according to some embodiments of the present disclosure;

FIG. 7 is a schematic diagram of a classification model for determininga target classification for a pulmonary nodule according to someembodiments of the present disclosure;

FIG. 8 includes a flowchart illustrating an exemplary process fordetermining a target classification of a structure of a subject in animage according to some embodiments of the present disclosure;

FIG. 9 includes a schematic diagram illustrating the determination of atarget classification of a structure of a subject in an image accordingto some embodiments of the present disclosure; and

FIG. 10 is a schematic diagram of an exemplary computing apparatusaccording to some embodiments of the present disclosure.

DETAILED DESCRIPTION

In the following detailed description, numerous specific details are setforth by way of examples in order to provide a thorough understanding ofthe relevant disclosure. However, it should be apparent to those skilledin the art that the present disclosure may be practiced without suchdetails. In other instances, well known methods, procedures, systems,components, and/or circuitry have been described at a relativelyhigh-level, without detail, in order to avoid unnecessarily obscuringaspects of the present disclosure. Various modifications to thedisclosed embodiments will be readily apparent to those skilled in theart, and the general principles defined herein may be applied to otherembodiments and applications without departing from the spirit and scopeof the present disclosure. Thus, the present disclosure is not limitedto the embodiments shown, but to be accorded the widest scope consistentwith the claims.

It will be understood that the term “system,” “engine,” “unit,”“module,” and/or “block” used herein are one method to distinguishdifferent components, elements, parts, section or assembly of differentlevel in ascending order. However, the terms may be displaced by anotherexpression if they may achieve the same purpose.

Generally, the word “module,” “unit,” or “block,” as used herein, refersto logic embodied in hardware or firmware, or to a collection ofsoftware instructions. A module, a unit, or a block described herein maybe implemented as software and/or hardware and may be stored in any typeof non-transitory computer-readable medium or another storage device. Insome embodiments, a software module/unit/block may be compiled andlinked into an executable program. It will be appreciated that softwaremodules can be callable from other modules/units/blocks or fromthemselves, and/or may be invoked in response to detected events orinterrupts. Software modules/units/blocks configured for execution oncomputing devices (e.g., processor 210 as illustrated in FIG. 2) may beprovided on a computer readable medium, such as a compact disc, adigital video disc, a flash drive, a magnetic disc, or any othertangible medium, or as a digital download (and can be originally storedin a compressed or installable format that needs installation,decompression, or decryption prior to execution). Such software code maybe stored, partially or fully, on a storage device of the executingcomputing device, for execution by the computing device. Softwareinstructions may be embedded in firmware, such as an erasableprogrammable read-only memory (EPROM). It will be further appreciatedthat hardware modules/units/blocks may be included of connected logiccomponents, such as gates and flip-flops, and/or can be included ofprogrammable units, such as programmable gate arrays or processors. Themodules/units/blocks or computing device functionality described hereinmay be implemented as software modules/units/blocks, but may berepresented in hardware or firmware. In general, themodules/units/blocks described herein refer to logicalmodules/units/blocks that may be combined with othermodules/units/blocks or divided into sub-modules/sub-units/sub-blocksdespite their physical organization or storage.

It will be understood that when a unit, engine, module or block isreferred to as being “on,” “connected to,” or “coupled to” another unit,engine, module, or block, it may be directly on, connected or coupledto, or communicate with the other unit, engine, module, or block, or anintervening unit, engine, module, or block may be present, unless thecontext clearly indicates otherwise. As used herein, the term “and/or”includes any and all combinations of one or more of the associatedlisted items.

The terminology used herein is for the purposes of describing particularexamples and embodiments only, and is not intended to be limiting. Asused herein, the singular forms “a,” “an,” and “the” may be intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “include” and/or“comprise,” when used in this disclosure, specify the presence ofintegers, devices, behaviors, stated features, steps, elements,operations, and/or components, but do not exclude the presence oraddition of one or more other integers, devices, behaviors, features,steps, elements, operations, components, and/or groups thereof.

Provided herein are systems and components for non-invasive imagingand/or treatment, such as for disease diagnosis, treatment or researchpurposes. In some embodiments, the system may be a radiation therapysystem, a computed tomography (CT) system, an emission computedtomography (ECT) system, an X-ray photography system, a positronemission tomography (PET) system, or the like, or any combinationthereof. For illustration purposes, the disclosure describes systems andmethods for radiation therapy. The term “image” used in this disclosuremay refer to a 2D image, a 3D image, or a 4D image. In some embodiments,the term “image” may refer to an image of a region, e.g., a region ofinterest (ROI), of a patient. The term “region of interest” or “ROI”used in this disclosure may refer to a part of an image along a line, intwo spatial dimensions, in three spatial dimensions, or any of theproceeding as they evolve as a function of time. The image may be a CTimage, PET image, an MR image, a fluoroscopy image, an ultrasound image,etc. This is not intended to limit the scope of the present disclosure.For persons having ordinary skills in the art, a certain number ofvariations, changes, and/or modifications may be deduced under theguidance of the present disclosure. Those variations, changes, and/ormodifications do not depart from the scope of the present disclosure.

According to an aspect of the present disclosure, a method fordetermining a target classification of a structure in an image may beprovided. The target classification may be determined from a pluralityof candidate classifications using a classification model. Theclassification model may be trained according to a focal loss function.At least one weight of the focal loss function each of which correspondsto one of the plurality of candidate classifications being differentfrom weights of the focal loss function corresponding to the remainderof the plurality of candidate classifications. By adjusting the weightscorresponding to the plurality of candidate classifications, negativeeffects induced by imbalance of samples corresponding to differentcandidate classifications on the classification model may be reduced,and a more robust classification model may be obtained. The use of thefocal loss function as the objective function may improve theperformance of the classification model. The classification model mayinclude a classification backbone network, a segmentation network, and adensity network. By using the segmentation network and the densitynetwork, the density and the size of the structure may be taken intoconsideration in the determination of the target classification of thestructure.

FIG. 1 is a schematic diagram illustrating an exemplary imaging system100 according to some embodiments of the present disclosure. This isunderstood that the systems and methods for determining a targetclassification of a structure of a subject are also applicable in othersystems, e.g., a treatment system. The following descriptions areprovided, unless otherwise stated expressly, with reference to animaging system for illustration purposes and not intended to belimiting. As illustrated, the imaging system 100 may include an imagingscanner 110, a processing device 120, a storage device 130, one or moreterminals 140, and a network 150. The components in the imaging system100 may be connected in various ways. Merely by way of example, asillustrated in FIG. 1, the imaging scanner 110 may be connected to theprocessing device 120 through the network 150. As another example, theimaging scanner 110 may be connected with the processing device 120directly as indicated by the bi-directional arrow in dotted lineslinking the imaging scanner 110 and the processing device 120. As afurther example, the storage device 130 may be connected with theprocessing device 120 directly (not shown in FIG. 1) or through thenetwork 150. As still a further example, one or more terminal(s) 140 maybe connected with the processing device 120 directly (as indicated bythe bi-directional arrow in dotted lines linking the terminal(s) 140 andthe processing device 120) or through the network 150.

The imaging scanner 110 may scan a subject or a portion thereof that islocated within its detection region, and generate imaging signalsrelating to the (part of) subject. In the present disclosure, the terms“subject” and “object” are used interchangeably. In some embodiments,the subject may include a body, a substance, or the like, or acombination thereof. In some embodiments, the subject may include aspecific portion of a body, such as the head, the thorax, the abdomen,or the like, or a combination thereof. In some embodiments, the subjectmay include a specific organ, such as the heart, the esophagus, thetrachea, the bronchus, the stomach, the gallbladder, the smallintestine, the colon, the bladder, the ureter, the uterus, the fallopiantube, etc. In some embodiments, the imaging scanner 110 may include acomputed tomography (CT) scanner, a positron emission computedtomography (PET) scanner, a single-photon emission computed tomography(SPECT) scanner, a magnetic resonance (MR) scanner, an ultrasonicscanner, an emission computed tomography (ECT) scanner, or the like. Insome embodiment, the imaging scanner 110 may be a multi-modality deviceincluding two or more scanners listed above. For example, the imagingscanner 110 may be a PET-CT scanner, a PET-MR scanner, etc.

Merely for illustration purposes, a PET-CT scanner may be provided as anexample for better understanding the imaging scanner 110, which is notintended to limit the scope of the present disclosure. The PET-CT mayinclude a gantry 111, a detecting region 112, and a bed 113. The gantry111 may support one or more radiation sources and/or detectors (notshown). A subject may be placed on the bed 113 for CT scan and/or PETscan. The PET-CT scanner may combine a CT scanner with a PET scanner.When the imaging scanner 110 performs a CT scan, a radiation source mayemit radioactive rays to the subject, and one or more detectors maydetect radiation rays emitted from the detecting region 112. Theradiation rays emitted from the detecting region 112 may be used togenerate CT data (also referred to as CT imaging information). The oneor more detectors used in CT scan may include a scintillation detector(e.g., a cesium iodide detector), a gas detector, etc.

To prepare for a PET scan, a radionuclide (also referred to as “PETtracer” or “PET tracer molecules”) may be introduced into the subject.The PET tracer may emit positrons in the detecting region 112 when itdecays. An annihilation (also referred to as “annihilation event” or“coincidence event”) may occur when a positron collides with anelectron. The annihilation may produce two gamma photons, which maytravel in opposite directions. The line connecting the detector unitsthat detecting the two gamma photons may be defined as a “line ofresponse (LOR).” One or more detector set on the gantry 111 may detectthe annihilation events (e.g., gamma photons) emitted from the detectingregion 112. The annihilation events emitted from the detecting region112 may be used to generate PET data (also referred to as PET imaginginformation). In some embodiments, the one or more detectors used in thePET scan may be different from detectors used in the CT scan. In someembodiments, the one or more detectors used in the PET scan may includecrystal elements and photomultiplier tubes (PMT).

The processing device 120 may process data and/or information obtainedand/or retrieve from the imaging scanner 110, the terminal(s) 140, thestorage device 130 and/or other storage devices. For example, theprocessing device 120 may obtain image data from the imaging scanner110, and reconstruct an image of the subject based on the image data. Asanother example, the processing device 120 may automatically determine,by inputting the image into an identification model, a targetclassification of a structure of the subject in the image (e.g., amalignant pulmonary nodule or a benign pulmonary nodule). As a furtherexample, the processing device 120 may train a classification modelbased on a plurality of training samples. In some embodiments, theprocessing device 120 may be a single server or a server group. Theserver group may be centralized or distributed. In some embodiments, theprocessing device 120 may be local or remote. For example, theprocessing device 120 may access information and/or data stored in theimaging scanner 110, the terminal(s) 140, and/or the storage device 130via the network 150. As another example, the processing device 120 maybe directly connected with the imaging scanner 110, the terminal(s) 140,and/or the storage device 130 to access stored information and/or data.In some embodiments, the processing device 120 may be implemented on acloud platform. Merely by way of example, the cloud platform may includea private cloud, a public cloud, a hybrid cloud, a community cloud, adistributed cloud, an inter-cloud, a multi-cloud, or the like, or anycombination thereof. In some embodiments, the processing device 120 maybe implemented on a computing apparatus 200 having one or morecomponents illustrated in FIG. 2 in the present disclosure.

The storage device 130 may store data and/or instructions. In someembodiments, the storage device 130 may store data obtained from theimaging scanner 110, the terminal(s) 140, and/or the processing device120. For example, the storage device 130 may store scanning data,signals, images, videos, algorithms, texts, instructions, program codes,etc. In some embodiments, the storage device 130 may store data and/orinstructions that the processing device 120 may execute or use toperform exemplary methods described in the present disclosure. In someembodiments, the storage device 130 may include a mass storage device, aremovable storage device, a volatile read-and-write memory, a read-onlymemory (ROM), or the like, or any combination thereof. Exemplary massstorage may include a magnetic disk, an optical disk, a solid-statedrive, etc. Exemplary removable storage may include a flash drive, afloppy disk, an optical disk, a memory card, a zip disk, a magnetictape, etc. Exemplary volatile read-and-write memories may include arandom access memory (RAM). Exemplary RAM may include a dynamic RAM(DRAM), a double date rate synchronous dynamic RAM (DDR SDRAM), a staticRAM (SRAM), a thyristor RAM (T-RAM), and a zero-capacitor RAM (Z-RAM),etc. Exemplary ROM may include a mask ROM (MROM), a programmable ROM(PROM), an erasable programmable ROM (PEROM), an electrically erasableprogrammable ROM (EEPROM), a compact disk ROM (CD-ROM), and a digitalversatile disk ROM, etc. In some embodiments, the storage device 130 maybe implemented on a cloud platform. Merely by way of example, the cloudplatform may include a private cloud, a public cloud, a hybrid cloud, acommunity cloud, a distributed cloud, an inter-cloud, a multi-cloud, orthe like, or any combination thereof.

In some embodiments, the storage device 130 may be connected with thenetwork 150 to communicate with one or more components of the imagingsystem 100 (e.g., the processing device 120, the terminal(s) 140, etc.).One or more components of the imaging system 100 may access the data orinstructions stored in the storage device 130 via the network 150. Insome embodiments, the storage device 130 may be directly connected withor communicate with one or more components of the imaging system 100(e.g., the processing device 120, the terminal(s) 140, etc.). In someembodiments, the storage device 130 may be part of the processing device120.

The terminal(s) 140 may include a mobile device 140-1, a tablet computer140-2, a laptop computer 140-3, or the like, or any combination thereof.In some embodiments, the mobile device 140-1 may include a smart homedevice, a wearable device, a smart mobile device, a virtual realitydevice, an augmented reality device, or the like, or any combinationthereof. In some embodiments, the smart home device may include a smartlighting device, a control device of an intelligent electricalapparatus, a smart monitoring device, a smart television, a smart videocamera, an interphone, or the like, or any combination thereof. In someembodiments, the wearable device may include a smart bracelet, smartfootgear, a pair of smart glasses, a smart helmet, a smartwatch, smartclothing, a smart backpack, a smart accessory, or the like, or anycombination thereof. In some embodiments, the smart mobile device mayinclude a smartphone, a personal digital assistant (PDA), a gamingdevice, a navigation device, a point of sale (POS) device, or the like,or any combination thereof. In some embodiments, the virtual realitydevice and/or the augmented reality device may include a virtual realityhelmet, a virtual reality glass, a virtual reality patch, an augmentedreality helmet, an augmented reality glass, an augmented reality patch,or the like, or any combination thereof. For example, the virtualreality device and/or the augmented reality device may include a GoogleGlass, an Oculus Rift, a Hololens, a Gear VR, etc. In some embodiments,the terminal(s) 140 may remotely operate the imaging scanner 110. Insome embodiments, the terminal(s) 140 may operate the imaging scanner110 via a wireless connection. In some embodiments, the terminal(s) 140may receive information and/or instructions inputted by a user, and sendthe received information and/or instructions to the imaging scanner 110or the processing device 120 via the network 150. In some embodiments,the terminal(s) 140 may receive data and/or information from theprocessing device 120. In some embodiments, the terminal(s) 140 may bepart of the processing device 120. In some embodiments, the terminal(s)140 may be omitted.

The network 150 may include any suitable network that can facilitate theexchange of information and/or data for the imaging system 100. In someembodiments, one or more components of the imaging system 100 (e.g., theimaging scanner 110, the terminal(s) 140, the processing device 120, orthe storage device 130) may communicate information and/or data with oneor more other components of the imaging system 100 via the network 150.In some embodiments, the network 150 may be any type of wired orwireless network, or a combination thereof. The network 150 may beand/or include a public network (e.g., the Internet), a private network(e.g., a local area network (LAN), a wide area network (WAN)), etc.), awired network (e.g., an Ethernet network), a wireless network (e.g., an802.11 network, a Wi-Fi network, etc.), a cellular network (e.g., a LongTerm Evolution (LTE) network), a frame relay network, a virtual privatenetwork (“VPN”), a satellite network, a telephone network, routers,hubs, switches, server computers, and/or any combination thereof. Merelyby way of example, the network 150 may include a cable network, awireline network, a fiber-optic network, a telecommunications network,an intranet, a wireless local area network (WLAN), a metropolitan areanetwork (MAN), a public telephone switched network (PSTN), a Bluetooth™network, a ZigBee™ network, a near field communication (NFC) network, orthe like, or any combination thereof. In some embodiments, the network150 may include one or more network access points. For example, thenetwork 150 may include wired and/or wireless network access points suchas base stations and/or internet exchange points through which one ormore components of the imaging system 100 may be connected with thenetwork 150 to exchange data and/or information.

It should be noted that the above description of the imaging system 100is merely provided for the purposes of illustration, not intended tolimit the scope of the present disclosure. For persons having ordinaryskills in the art, components contained in the imaging system 100 may becombined or adjusted in various ways, or connected with other componentsas sub-systems, and various variations and modifications may beconducted under the teaching of the present disclosure. However, thosevariations and modifications may not depart the spirit and scope of thisdisclosure. For example, the imaging scanner 110 may be a standalonedevice external to the imaging system 100, and the imaging system 100may be connected to or in communication with the imaging scanner 110 viathe network 150. All such modifications are within the protection scopeof the present disclosure.

FIG. 2 is a schematic diagram illustrating hardware and/or softwarecomponents of an exemplary computing apparatus 200 on which theprocessing device 120 may be implemented according to some embodimentsof the present disclosure. As illustrated in FIG. 2, the computingapparatus 200 may include a processor 210, a storage 220, aninput/output (I/O) 230, and a communication port 240.

The processor 210 may execute computer instructions (program code) andperform functions of the processing device 120 in accordance withtechniques described herein. The computer instructions may include, forexample, routines, programs, objects, components, signals, datastructures, procedures, modules, and functions, which perform particularfunctions described herein. For example, the processor 210 may processdata obtained from the imaging scanner 110, the terminal(s) 140, thestorage device 130, and/or any other component of the imaging system100. Specifically, the processor 210 may process image data obtainedfrom the imaging scanner 110. For example, the processor 210 maygenerate an image based on the image data and identify a structure of atarget classification from the image. In some embodiments, the image maybe stored in the storage device 130, the storage 220, etc. In someembodiments, the image may be displayed on a display device by the I/O230. In some embodiments, the processor 210 may perform instructionsobtained from the terminal(s) 140. In some embodiments, the processor210 may include one or more hardware processors, such as amicrocontroller, a microprocessor, a reduced instruction set computer(RISC), an application specific integrated circuits (ASICs), anapplication-specific instruction-set processor (ASIP), a centralprocessing unit (CPU), a graphics processing unit (GPU), a physicsprocessing unit (PPU), a microcontroller unit, a digital signalprocessor (DSP), a field programmable gate array (FPGA), an advancedRISC machine (ARM), a programmable logic device (PLD), any circuit orprocessor capable of executing one or more functions, or the like, orany combinations thereof.

Merely for illustration, only one processor is described in thecomputing apparatus 200. However, it should be noted that the computingapparatus 200 in the present disclosure may also include multipleprocessors. Thus operations and/or method steps that are performed byone processor as described in the present disclosure may also be jointlyor separately performed by the multiple processors. For example, if inthe present disclosure the processor of the computing apparatus 200executes both operation A and operation B, it should be understood thatoperation A and operation B may also be performed by two or moredifferent processors jointly or separately in the computing apparatus200 (e.g., a first processor executes operation A and a second processorexecutes operation B, or the first and second processors jointly executeoperations A and B).

The storage 220 may store data/information obtained from the imagingscanner 110, the terminal(s) 140, the storage device 130, or any othercomponent of the imaging system 100. In some embodiments, the storage220 may include a mass storage device, a removable storage device, avolatile read-and-write memory, a read-only memory (ROM), or the like,or any combination thereof. For example, the mass storage may include amagnetic disk, an optical disk, a solid-state drive, etc. The removablestorage may include a flash drive, a floppy disk, an optical disk, amemory card, a zip disk, a magnetic tape, etc. The volatileread-and-write memory may include a random access memory (RAM). The RAMmay include a dynamic RAM (DRAM), a double date rate synchronous dynamicRAM (DDR SDRAM), a static RAM (SRAM), a thyristor RAM (T-RAM), and azero-capacitor RAM (Z-RAM), etc. The ROM may include a mask ROM (MROM),a programmable ROM (PROM), an erasable programmable ROM (PEROM), anelectrically erasable programmable ROM (EEPROM), a compact disk ROM(CD-ROM), and a digital versatile disk ROM, etc. In some embodiments,the storage 220 may store one or more programs and/or instructions toperform exemplary methods described in the present disclosure. Forexample, the storage 220 may store a program for the processing device120 for determining a target classification of a structure of a subject.

The I/O 230 may input or output signals, data, and/or information. Insome embodiments, the I/O 230 may enable user interaction with theprocessing device 120. In some embodiments, the I/O 230 may include aninput device and an output device. Exemplary input devices may include akeyboard, a mouse, a touch screen, a microphone, or the like, or acombination thereof. Exemplary output devices may include a displaydevice, a loudspeaker, a printer, a projector, or the like, or acombination thereof. Exemplary display devices may include a liquidcrystal display (LCD), a light-emitting diode (LED)-based display, aflat panel display, a curved screen, a television device, a cathode raytube (CRT), or the like, or a combination thereof.

The communication port 240 may be connected with a network (e.g., thenetwork 150) to facilitate data communications. The communication port240 may establish connections between the processing device 120 and theimaging scanner 110, the terminal(s) 140, or the storage device 130. Theconnection may be a wired connection, a wireless connection, or acombination of both that enables data transmission and reception. Thewired connection may include an electrical cable, an optical cable, atelephone wire, or the like, or any combination thereof. The wirelessconnection may include Bluetooth, Wi-Fi, WiMax, WLAN, ZigBee, mobilenetwork (e.g., 3G, 4G, 5G, etc.), or the like, or a combination thereof.In some embodiments, the communication port 240 may be a standardizedcommunication port, such as RS232, RS485, etc. In some embodiments, thecommunication port 240 may be a specially designed communication port.For example, the communication port 240 may be designed in accordancewith the digital imaging and communications in medicine (DICOM)protocol.

FIG. 3 is a schematic diagram illustrating hardware and/or softwarecomponents of an exemplary mobile device 300 according to someembodiments of the present disclosure. As illustrated in FIG. 3, themobile device 300 may include a communication platform 310, a display320, a graphics processing unit (GPU) 330, a central processing unit(CPU) 340, an I/O 350, a memory 370, and a storage 390. In someembodiments, any other suitable component, including but not limited toa system bus or a controller (not shown), may also be included in themobile device 300. In some embodiments, a mobile operating system 360(e.g., iOS, Android, Windows Phone, etc.) and one or more applications380 may be loaded into the memory 370 from the storage 390 in order tobe executed by the CPU 340. The applications 380 may include a browseror any other suitable mobile apps for receiving and renderinginformation relating to data processing or other information from theprocessing device 120. User interactions with the information stream maybe achieved via the I/O 350 and provided to the processing device 120and/or other components of the imaging system 100 via the network 150.

To implement various modules, units, and their functionalities describedin the present disclosure, computer hardware platforms may be used asthe hardware platform(s) for one or more of the elements describedherein. The hardware elements, operating systems and programminglanguages of such computers are conventional in nature, and it ispresumed that those skilled in the art are adequately familiar therewithto adapt those technologies to generate an imaging report as describedherein. A computer with user interface elements may be used to implementa personal computer (PC) or another type of work station or terminaldevice, although a computer may also act as a server if appropriatelyprogrammed. It is believed that those skilled in the art are familiarwith the structure, programming and general operation of such computerequipment and as a result, the drawings should be self-explanatory.

FIG. 4 is a block diagram illustrating an exemplary processing deviceaccording to some embodiments of the present disclosure. The processingdevice 120 may include an obtaining module 410, a pre-processing module420, a classification determination module 430, and a model trainingmodule 440. One or more of the modules of the processing device 120 maybe interconnected. The connection(s) may be wireless or wired. At leasta portion of the processing device 120 may be implemented on a computingapparatus as illustrated in FIG. 2 or a mobile device as illustrated inFIG. 3.

The obtaining module 410 may obtain data. The obtaining module 410 mayobtain data from the imaging scanner 110, the processing device 120, thestorage device 130, the terminal 140, or any devices or componentscapable of storing data via the network 150. For example, the obtainingmodule 410 may obtain data from a medical cloud data center (not shown)via the network 150. The obtain data may include scanning data (e.g.,original image data), processed results (e.g., image data), userinstructions, algorithms, models (e.g., a classification model), programcodes, or the like, or a combination thereof. In some embodiments, theobtaining module 410 may obtain image data representing a structure of asubject. The obtaining module 410 may obtain the image data from amedical imaging system, such as a magnetic resonance imaging (MRI)system, a computed tomography (CT) system, a digital X-ray imagingsystem, an ultrasound imaging system, a positron emission computedtomography (PET) system, a PET-MR system, a PET-CT system, etc. In someembodiments, the structure of the subject may be a specific portion of abody of the subject, such as the head, the thorax, the abdomen, or thelike, or a combination thereof. In some embodiments, the structure ofthe subject may be a specific organ of the subject, such as lungs, theheart, the esophagus, the trachea, the bronchus, the stomach, thegallbladder, the small intestine, the colon, the bladder, the ureter,the uterus, the fallopian tube, etc.

The obtaining module 410 may transmit the obtained data to a storagedevice (e.g., the storage device 130, etc.) for storage. In someembodiments, the obtaining module 410 may transmit the obtained data toa computing device (including, for example, pre-processing module 420, aclassification determination module 430, etc.) for processing.

The pre-processing module 420 may perform pre-processing operations ondata. In some embodiments, the pre-processing module 420 may performpre-processing operations on original image data. The original imagedata may refer to image data acquired by the imaging scanner 110. Insome embodiments, the original image data may be pre-processed todetermine the image data representing the structure of the subject.

In some embodiments, pre-processing module 420 may performpre-processing operations on the original image data using a presetimage pre-processing algorithm. In some embodiments, the presetpre-processing operation may include re-sampling, according to aresampling resolution, the original image data to generate a resampledimage. The resampling resolution may be preset, for example, by a user,according to default settings of the imaging system 100, etc. In someembodiments, the pre-processing operation may include segmenting theresampled image into image crops according to a center of the structure.In some embodiments, the pre-processing operation may also includenormalizing the image crops using a preset normalization algorithm toobtain the image data to be input into the classification model. In someembodiments, the pre-processing operation may also include a denoisingoperation for removing or reducing noise or errors in the image data.

The classification determination module 430 may determine a targetclassification of the structure of the subject. In some embodiments, thetarget classification of the structure may be determined based onprobabilities of a plurality of candidate classifications. The pluralityof candidate classifications of the structure and their respectiveprobabilities may be determined by inputting the image data into aclassification model. The classification model may trained according toa focal loss function, at least one weight of the focal loss functioneach of which corresponds to one of the plurality of candidateclassifications being different from weights of the focal loss functioncorresponding to the remainder of the plurality of candidateclassifications. In some embodiments, the target classification of thepulmonary nodule may be determined by identifying a candidateclassification of the structure corresponding to a largest probabilityamong the probabilities of the plurality of candidate classifications.The identified candidate classification may be designated as the targetclassification of the structure.

The model training module 440 may train a model. In some embodiments,the model training module 440 may train a preliminary classificationmodel. In some embodiments, the preliminary classification model may beany type of network model that is to be trained as the classificationmodel. In some embodiments, the preliminary classification model mayinclude a preliminary classification backbone network, a preliminarysegmentation network, and a preliminary density network. Merely forillustration purposes, the preliminary classification backbone networkmay include a plurality of downsampling layers. In some embodiments,each downsampling layer may include a convolution layer, a batchnormalization (BN) layer, and a rectified linear units (ReLU) layer. Insome embodiments, the structure of the preliminary density network maybe the same as or similar to the structure of the preliminaryclassification backbone network. The preliminary segmentation networkmay include a fully convolutional network (FCN), a SegNet (semanticsegmentation network), a CRFasRNN (conditional random fields asrecurrent neural network) model, a PSPNet, a ParseNet (Pyramid SceneParsing Network), an ENet (efficient neural network), a RefineNet, orthe like, or any combination thereof. In some embodiments, thepreliminary classification model may further include a fully connectedlayer and a softmax layer.

In some embodiments, a training sample set may be generated based onhistorical image data. Structures may be labeled in the historical imagedata. Target classifications corresponding to the labeled structures inthe historical image data may also be obtained. The labeled structuresand the target classifications may constitute the training sample set.The model training module 440 may train the preliminary classificationmodel using the training sample set to obtain a trained classificationmodel.

In some embodiments, the preliminary classification model to be trainedmay include one or more model parameters. Exemplary model parameters mayinclude the number (or count) of layers, the number (or count) of nodes,or the like, or any combination thereof. Before training, thepreliminary classification model may have one or more initial parametervalues of the model parameter(s). In the training of the preliminarymodel, the value(s) of the model parameter(s) of the preliminary modelmay be updated.

In some embodiments, the training of the preliminary classificationmodel may include one or more iterations to iteratively update the modelparameters of the preliminary model until a termination condition issatisfied in a certain iteration. Exemplary termination conditions maybe that the value of an objective function (i.e., loss function)obtained in the certain iteration is less than a threshold value, that acertain count of iterations have been performed, that the objectivefunction converges such that the difference of the values of theobjective function obtained in a previous iteration and the currentiteration is within a threshold value, etc. Exemplary objectivefunctions may include a focal loss function, a log loss function, across-entropy loss function, a Dice loss function, etc.

In some embodiments, a focal loss function may be used as the objectivefunction. A loss value may be determined by comparing an output of theclassification model with a target classification of a structurecorresponding to historical image data input into the classificationmodel (i.e., the target classification and the historical image data mayconstitute a training sample pair). In some embodiments, the preliminaryclassification model may be trained in a plurality of iterations. Duringthe plurality of iterations, an optimal solution of the focal lossfunction may be obtained.

In some embodiments, each of the plurality of candidate classificationsmay correspond to a weight in the focal loss function. At least oneweight of the focal loss function each of which corresponds to one ofthe plurality of candidate classifications being different from weightsof the focal loss function corresponding to the remainder of theplurality of candidate classifications. For example, a smaller weightmay be assigned to a benign pulmonary nodule, and a larger weight may beassigned to a malignant pulmonary nodule. By adjusting the weightscorresponding to the plurality of candidate classifications, negativeeffects induced by imbalance of samples corresponding to differentcandidate classifications on the classification model may be reduced,and a more robust classification model may be obtained. In the presentdisclosure, the terms “classification model” and “trained classificationmodel” may be used interchangeably. The use of the focal loss functionas the objective function may improve the performance of theclassification model.

It should be noted that the above descriptions of the processing device120 are provided for the purposes of illustration, and not intended tolimit the scope of the present disclosure. For persons having ordinaryskills in the art, various modifications and changes in the forms anddetails of the application of the above method and system may occurwithout departing from the principles of the present disclosure. In someembodiments, the processing device 120 may include one or more othermodules. In some embodiments, two or more units in the processing device120 may form one module. However, those variations and modificationsalso fall within the scope of the present disclosure.

FIG. 5 includes a flowchart illustrating an exemplary process fordetermining a target classification of a structure of a subject in animage according to some embodiments of the present disclosure. In someembodiments, at least a portion of the process 500 may be performed bythe processing device 120 (e.g., implemented in the computing apparatus200 shown in FIG. 2, the processing device illustrated in FIG. 4). Insome embodiments, at least a portion of the process 500 may be performedby a terminal device (e.g., the mobile device 300 shown in FIG. 3)embodying software and/or hardware.

In 510, image data representing a structure of a subject may beobtained. The image data may be obtained by the obtaining module 410.

The image data may be obtained from a medical imaging system, such as amagnetic resonance imaging (MRI) system, a computed tomography (CT)system, a digital X-ray imaging system, an ultrasound imaging system, apositron emission computed tomography (PET) system, a PET-MR system, aPET-CT system, etc. In some embodiments, an image may be reconstructedbased on the image data. The image may be reconstructed using anysuitable algorithm corresponding to a type of the medical imagingsystem. In some embodiments, the structure of the subject may be aspecific portion of a body of the subject, such as the head, the thorax,the abdomen, or the like, or a combination thereof. In some embodiments,the structure of the subject may be a specific organ of the subject,such as lungs, the heart, the esophagus, the trachea, the bronchus, thestomach, the gallbladder, the small intestine, the colon, the bladder,the ureter, the uterus, the fallopian tube, etc.

In some embodiments, the image may be an image of the lungs of thesubject. Pulmonary nodules on the lungs of the subject may be identifiedin the image. For example, features of pulmonary nodules may beextracted from images of lungs of a plurality of patients. The extractedfeatures of the pulmonary nodules on the lungs of the plurality ofpatients may be stored in, for example, a feature library. The featuresin the feature library may be compared with features of structures onthe lungs of the subject. If features of a structure on a lung of thesubject matches certain features in the feature library, the structuremay be identified as a pulmonary nodule.

In some embodiments, if it is needed to determined a targetclassification (e.g., a benign pulmonary nodule or a malignant pulmonarynodule) of a pulmonary nodule according to the image of the lungs of thesubject, the pulmonary nodule in the image may be labeled, e.g., bymarking a center point of the pulmonary nodule with a specificidentifier. In some embodiments, the center point of the pulmonarynodule may be labeled manually by a user (e.g., a doctor, a technician,etc.). In some embodiments, the labeling of the center point of thepulmonary nodule may be automated, e.g., using a processing device(e.g., a detection device). In some embodiments, the detection devicemay be implemented by a detection model. The center point of thepulmonary nodule may be labeled by inputting the image of the subjectinto the detection model. Exemplary detection models may include a deepbelief network (DBN), a stacked auto-encoders (SAE), a logisticregression (LR) model, a support vector machine (SVM) model, a decisiontree model, a naive Bayesian model, a random forest model, or arestricted Boltzmann machine (RBM), a gradient boosting decision tree(GBDT) model, a LambdaMART model, an adaptive boosting model, arecurrent neural network (RNN) model, a convolutional network model, ahidden Markov model, a perceptron neural network model, a Hopfieldnetwork model, or the like, or any combination thereof. The image of thelungs of the subject with the marked pulmonary nodule may be input intoa classification model to determine the target classification of thepulmonary nodule. It should be noted that the way that the center pointof the pulmonary nodule is labeled is not limited in the presentdisclosure.

In 520, a plurality of candidate classifications of the structure andtheir respective probabilities may be determined by inputting the imagedata into a classification model, wherein the classification model istrained according to a focal loss function, at least one weight of thefocal loss function each of which corresponds to one of the plurality ofcandidate classifications being different from weights of the focal lossfunction corresponding to the remainder of the plurality of candidateclassifications. In some embodiments, the classification model may betrained by the model training module 440.

As used herein, the plurality of candidate classifications of thestructure refers to all possible types of the structure. For example,since a pulmonary nodule may be a benign pulmonary nodule, a suspectedpulmonary nodule, or a malignant pulmonary nodule, candidateclassifications of a pulmonary nodule may include benign pulmonarynodule, suspected pulmonary nodule, and malignant pulmonary nodule. Insome embodiments, different candidate classifications of the structuremay have different features. The features of the structure may include asize, a density, a shape, etc. Taking a size of a pulmonary nodule as anexample, in general, the larger the size of the pulmonary nodule is, thegreater the possibility that the pulmonary nodule is a malignantpulmonary nodule will be. Most pulmonary nodules having diameters lessthan 5 millimeters may be benign pulmonary nodules, and most pulmonarynodules having diameters larger than 20 millimeters may be malignantpulmonary nodules. A pulmonary nodule having a diameter larger than 5millimeters and smaller than 20 millimeters may be determined as asuspected pulmonary nodule. In addition, the candidate classificationsof a structure may also relate to the density of the structure. Taking adensity of a pulmonary nodule as an example, the pulmonary nodule may besorted, according to the density of the pulmonary nodule, as aground-glass nodule, a calcified nodule, a solid nodule, or a mixednodule. Calcified nodules may substantially be benign pulmonary nodules,and ground-glass nodules may substantially be malignant pulmonarynodules.

Merely for illustration purposes, a classification model may be providedin combination with a pulmonary nodule. The classification model mayinclude a classification backbone network, a segmentation network, and adensity network. The classification backbone network may determine abackbone feature of the structure (e.g., features corresponding to aplurality of candidate classifications of the structure). Thesegmentation network may determine a segmentation feature (e.g., size)of the structure. The density network may determine a density feature(e.g., density) of the structure. In some embodiments, theclassification model may further include a fully connected layer and asoftmax layer. Details of the classification model may be describedelsewhere in the present disclosure, see, e.g., FIGS. 6 and 7 and thedescriptions thereof.

In some embodiments, the image data obtained in 510 may be input intothe classification backbone network, the segmentation network, and thedensity network, respectively. The output of each of the classificationbackbone network, the segmentation network, and the density network maybe obtained. In some embodiments, the output of the classificationbackbone network, the segmentation network, and the density network maybe or include a probability corresponding to each of the plurality ofcandidate classifications. The target classification of the structuremay be determined based on the output of the classification backbonenetwork, the segmentation network, and the density network.

In some embodiments, each of the plurality of candidate classificationsmay correspond to a weight in the focal loss function. At least oneweight of the focal loss function each of which corresponds to one ofthe plurality of candidate classifications being different from weightsof the focal loss function corresponding to the remainder of theplurality of candidate classifications. For example, a smaller weightmay be assigned to a benign pulmonary nodule, and a larger weight may beassigned to a malignant pulmonary nodule. By adjusting the weightscorresponding to the plurality of candidate classifications, negativeeffects induced by imbalance of samples corresponding to differentcandidate classifications on the classification model may be reduced,and a more robust classification model may be obtained. The use of thefocal loss function as the objective function may improve theperformance of the classification model.

In 530, a target classification of the structure may be determined basedon at least a part of the probabilities of the plurality of candidateclassifications. In some embodiments, the target classification of thestructure may be determined by the classification determination module430.

In some embodiments, the output of the classification model may includea plurality of candidate classification of the structure and aprobability corresponding to each of the plurality of candidateclassifications. Since the classification model includes aclassification backbone network, a segmentation network, and a densitynetwork, three sets of candidate classifications of the structure andthree sets of corresponding probabilities may be determined by inputtingthe image data into the classification model. The target classificationof the structure may be determined based on at least two of the threesets of probabilities from the three sets of candidate classifications.For example, the target classification of the structure may bedetermined based on probabilities from the candidate classificationscorresponding to the classification backbone network and thesegmentation network. As another example, the target classification ofthe structure may be determined based on probabilities from thecandidate classifications corresponding to the classification backbonenetwork and the density network. Since three networks including theclassification backbone network, the segmentation network, and thedensity network are used in the classification model, the accuracy ofthe determation of the target classification of the structure may beimproved, and the need for human intervention on the determination ofthe target classification of the structure may be reduced.

Merely by ways of example, for a pulmonary nodule on a lung of asubject, there may be three candidate classifications of the pulmonarynodule, e.g., benign pulmonary nodule, suspected pulmonary nodule, andmalignant pulmonary nodule. The probability corresponding to each of thethree candidate classifications may represent a probality that thepulmonary nodule is a benignancy, a probality that the pulmonary noduleis a suspected malignancy, and a probality that the pulmonary nodule isa malignancy, respectively. In some embodiments, the targetclassification of the pulmonary nodule may be determined by identifyinga candidate classification of the structure corresponding to a largestprobability among the probabilities of the plurality of candidateclassifications. The identified candidate classification may bedesignated as the target classification of the structure. For example,if the probability corresponding to a candidate classification of abenignancy is 0.73, the probability corresponding to a candidateclassification of a suspected malignancy is 0.35, and the probabilitycorresponding to a candidate classification of a malignancy is 0.27, itmay be determined that the pulmonary nodule in the image may be benign.

The operations for determining a target classification of a structure ofa subject in an image provided in the process 500 may include obtainingthe image data representing the structure of the subject, inputting theimage data into the classification model including the classificationbackbone network, the segmentation network, and the density network, soas to obtain the plurality of candidate classifications of the structureand their respective probabilities, and determine the targetclassification of the structure based on the probabilities of theplurality of candidate classifications. The target classification of thepulmonary nodule may be determined by combining the classificationbackbone network, the segmentation network and the density network,thereby improving the accuracy of the diagnosis of the pulmonary nodule.

In some embodiments, original image data may be obtained. The originalimage data may refer to image data acquired by the imaging scanner 110.In some embodiments, the original image data may be pre-processed todetermine the image data representing the structure of the subject.

In some embodiments, original image data obtained by different medicalimaging systems and/or using different imaging parameters may vary interms of resolution, distribution range of grey values, etc. In order toobtain image data satisfying requirements of the classification backbonenetwork, the segmentation network, and/or the density network,pre-processing operations may be performed on the original image data toobtain the image data. The pre-processing operations may be performed bythe pre-processing module 420.

In some embodiments, the original image data may be pre-processed usinga preset image pre-processing algorithm. In some embodiments, thepre-processing operation may include re-sampling, according to aresampling resolution, the original image data to generate a resampledimage. The resampling resolution may be preset, for example, by a user,according to default settings of the imaging system 100, etc. In someembodiments, the pre-processing operation may include segmenting theresampled image into image crops according to a center of the structure.In some embodiments, the pre-processing operation may also includenormalizing the image crops using a preset normalization algorithm toobtain the image data to be input into the classification model. In someembodiments, the pre-processing operation may also include a denoisingoperation for removing or reducing noise or errors in the image data.

Merely for illustration purposes, the original image data may representa lung of a subject. Center points of pulmonary nodules on the lung maybe labeled in the image data. The image data may be resampled, segmened,and/or normalized, to obtain pre-processed image data. For example, theimage data may be resampled to have a specified resolution. As anotherexample, the resampled image data may be segmented into image crops withpreset sizes (e.g., 64*64*64) according to a center of a pulmonarynodule (i.e., a center of an image crop coincides with the center of thepulmonary nodule). As a further example, the segmented image data may benormalized into a specified grayscale distribution range (e.g., 0-1). Insome embodiments, the image may further be denoised using a denoisingalgorithm.

FIG. 6 includes a flowchart illustrating an exemplary process fordetermining a target classification of a structure of a subject in animage according to some embodiments of the present disclosure. In someembodiments, at least a portion of the process 600 may be performed bythe processing device 120 (e.g., implemented in the computing apparatus200 shown in FIG. 2, the processing device illustrated in FIG. 4). Insome embodiments, at least a portion of the process 600 may be performedby a terminal device (e.g., the mobile device 300 shown in FIG. 3)embodying software and/or hardware.

In 610, image data representing a structure of a subject may beobtained. In some embodiments, the image data may be obtained by theobtaining module 410. In some embodiments, the operation 610 may be thesame as or similar to the operation 510 in the process 500 asillustrated in FIG. 5.

In 620, the image data may be input into a classification backbonenetwork, a segmentation network, and a density network, respectively.

In some embodiments, the classification backbone network, thesegmentation network, and the density network may be components of aclassification model. In some embodiments, the classification model maybe described in combination with the classification of a pulmonarynodule. The image data obtained in 610 may be input into theclassification backbone network, the segmentation network, and thedensity network, respectively.

The classification backbone network may determine a backbone feature ofthe structure (e.g., features corresponding to a plurality of candidateclassifications of the structure). Merely for illustration purposes, theclassification backbone network may include a plurality of (e.g., 2, 4,8, 10, etc.) downsampling layers. In some embodiments, each downsamplinglayer may include a convolution layer, a batch normalization (BN) layer,and a rectified linear units (ReLU) layer. The BN layer may beconfigured to receive and normalize an output of the convolutional layer(e.g., feature maps). It should be noted that the implementation of eachdownsampling layer may not be limited in the present disclosure. In someembodiments, each downsampling layer may include one or more DenseNetblocks and/or ResNet blocks. For example, each downsampling layer may beimplemented using a ResNet block. The ResNet block may facilitate abetter convergency for deep networks.

The density network may determine a density feature (e.g., density) ofthe structure. In some embodiments, the structure of the density networkmay be the same as or similar to the structure of the classificationbackbone network. For example, the density network may also include aplurality of downsampling layers.

The segmentation network may determine a segmentation feature (e.g.,size) of the structure. The segmentation network may include a fullyconvolutional network (FCN), a SegNet (semantic segmentation network), aCRFasRNN (conditional random fields as recurrent neural network) model,a PSPNet, a ParseNet (Pyramid Scene Parsing Network), an ENet (efficientneural network), a RefineNet, or the like, or any combination thereof.

In 630, a backbone feature output by the classification backbonenetwork, a segmentation feature output by the segmentation network, anda density feature output by the density network may be obtained.

In some embodiments, after the image data is input into theclassification backbone network, the segmentation network, and thedensity network, the backbone feature, the segmentation feature, and thedensity feature may be output from the classification backbone network,the segmentation network, and the density network, respectively. In someembodiments, the backbone feature, the segmentation feature, and thedensity feature may be expressed in forms of feature maps.

In 640, a plurality of candidate classifications and their respectiveprobabilities may be determined based on the backbone feature, thesegmentation feature, and the density feature.

In some embodiments, the plurality of candidate classifications andtheir respective probabilities may be generated by combining thebackbone feature output by the classification backbone network, thesegmentation feature output by the segmentation network, and the densityfeature output by the density network. In some embodiments, the backbonefeature, the segmentation feature, and the density feature may beexpressed in forms of feature maps. The feature maps may be combined toobtain a new feature map. The new feature map may be input into a fullyconnected layer and a softmax layer to determine the plurality ofcandidate classifications and their respective probabilities. The fullyconnected layer may connect to the classification backbone network, thesegmentation network, and the density network. The softmax layer may besubsequent to the fully connected layer. In some embodiments, the outputof the softmax layer may be expressed in the form of a probability map.The probability map may include the plurality of candidateclassifications and the probabilities corresponding to each candidateclassification.

In some embodiments, the backbone feature, the segmentation feature, andthe density feature may be combined to obtain an classification featureof the image data. The plurality of candidate classifications and theprobabilities corresponding to each candidate classification may bedetermined based on the classification feature. In some embodiments, thebackbone feature, the segmentation feature, and the density featureoutput by the classification backbone network, the segmentation network,and the density network, respectively, may be expressed in the form ofthree-dimensional feature maps. In some embodiments, before theclassification feature is determined, the three-dimensional feature mapof the backbone feature, the three-dimensional feature map of thesegmentation feature, and the three-dimensional feature map of thedensity feature may be converted into a one-dimensional backbone featurevector, a one-dimensional segmentation feature vector, and aone-dimensional density feature vector, respectively. The combination ofthe backbone feature, the segmentation feature, and the density featuremay be implemented based on the one-dimensional backbone feature vector,the one-dimensional segmentation feature vector, and the one-dimensionaldensity feature vector.

For example, the combination of the backbone feature, the segmentationfeature, and the density feature may be realized by converting thebackbone feature, the segmentation feature, and the density feature intoa one-dimensional backbone feature vector, a one-dimensionalsegmentation feature vector, and a one-dimensional density featurevector, respectively, and splicing the one-dimensional backbone featurevector, the one-dimensional segmentation feature vector, and theone-dimensional density feature vector to obtain a one-dimensionalclassification feature vector. The one-dimensional classificationfeature vector may be used to determine the plurality of candidateclassifications and the probabilities corresponding to each candidateclassification.

In some embodiments, the order in which the one-dimensional backbonefeature vector, the one-dimensional segmentation feature vector, and theone-dimensional density feature vector are spliced may not be limited inthe present disclosure. Merely by ways of example, if theone-dimensional backbone feature vector is [0, 1, 1, 5, 6, 8], theone-dimensional segmentation feature vector is [2, 3, 4], theone-dimensional segmentation feature vector is [7, 9, 5], theone-dimensional classification feature vector obtained by splicing theone-dimensional backbone feature vector, the one-dimensionalsegmentation feature vector, and the one-dimensional density featurevector may be [0, 1, 1, 5, 6, 8, 2, 3, 4, 7, 9, 5].

In 650, a target classification of the structure may be determined basedon at least a part of the probabilities of the plurality of candidateclassifications. In some embodiments, the target classification of thestructure may be determined by the classification determination module430. In some embodiments, the operation 650 may be the same as orsimilar to the operation 530 in the process 500 as illustrated in FIG.5.

FIG. 7 is a schematic diagram of a classification model for determininga target classification for a pulmonary nodule according to someembodiments of the present disclosure. The classification may include asegmentation network 720, a classification backbone network 730, adensity network 740, a fully connected layer 750, and a softmax layer760. As shown in FIG. 7, an image block 710 (e.g., including one or moreimages, each of which represents the pulmonary nodule(s) on a lung of asubject) may be obtained. The image block 710 may be input into thesegmentation network 720, the classification backbone network 730, andthe density network 740, respectively. The output of the segmentationnetwork 720, the classification backbone network 730, and the densitynetwork 740 may be feature maps. The feature maps may be combined todetermine a classification feature 770 of the image block 710. Theclassification feature 770 of the image block may be into the fullyconnected layer 750 and the softmax layer 760. The output of the softmaxlayer 760 may be a probability map. The probability map may include aplurality of candidate classifications of the pulmonary nodule (e.g.,benign pulmonary nodule, suspected pulmonary nodule, and malignantpulmonary nodule) and probabilities corresponding to each candidateclassification.

Technical solutions disclosed in the above embodiments of the presentdisclosure may provide the working process of the classification modelincluding, for example, inputting image data representing a structure ofa subject into a classification backbone network, a segmentationnetwork, and a density network, respectively, obtaining a backbonefeature output by the classification backbone network, a segmentationfeature output by the segmentation network, and a density feature outputby the density network, and determining a target classification of thestructure according to the backbone feature, the segmentation feature,and the density feature. In this way, a structure of a certain type(e.g., malignant pulmonary nodule) may be identified combining the size,the density, and other features of the structure, and the accuracy of amedical diagnosis may be improved.

FIG. 8 includes a flowchart illustrating an exemplary process fordetermining a target classification of a structure of a subject in animage according to some embodiments of the present disclosure. In someembodiments, at least a portion of the process 800 may be performed bythe processing device 120 (e.g., implemented in the computing apparatus200 shown in FIG. 2, the processing device illustrated in FIG. 4). Insome embodiments, at least a portion of the process 800 may be performedby a terminal device (e.g., the mobile device 300 shown in FIG. 3)embodying software and/or hardware.

In 810, a preliminary classification model and a training sample set maybe obtained.

In some embodiments, the preliminary classification model may be anytype of network model that is to be trained as the classification model.In some embodiments, the preliminary classification model may include apreliminary classification backbone network, a preliminary segmentationnetwork, and a preliminary density network. Merely for illustrationpurposes, the preliminary classification backbone network may include aplurality of downsampling layers. In some embodiments, each downsamplinglayer may include a convolution layer, a batch normalization (BN) layer,and a rectified linear units (ReLU) layer. In some embodiments, eachdownsampling layer may include one or more DenseNet blocks and/or ResNetblocks. For example, each downsampling layer may be implemented by aResNet block. The ResNet block may facilitate a better convergency fordeep networks. In some embodiments, the structure of the preliminarydensity network may be the same as or similar to the structure of thepreliminary classification backbone network. For example, thepreliminary density network may also include a plurality of (e.g., four)downsampling layers. The preliminary segmentation network may include afully convolutional network (FCN), a SegNet (semantic segmentationnetwork), a CRFasRNN (conditional random fields as recurrent neuralnetwork) model, a PSPNet, a ParseNet (Pyramid Scene Parsing Network), anENet (efficient neural network), a RefineNet, or the like, or anycombination thereof. In some embodiments, the preliminary classificationmodel may further include a fully connected layer and a softmax layer.

In some embodiments, the training sample set may be generated based onthe historical image data, the structures labeled in the image data, andthe target classifications corresponding to the structures in thehistorical image data. In some embodiments, the training sample set maybe previously generated and stored in a storage device (e.g., thestorage device 130, the storage 220, the storage 390, or an externalsource). For example, the training sample set may include historicalimage data representing structures of a plurality of subject generatedusing the imaging device 110, whereins center points of the structuresmay be labeled in the historical image data and target classificationscorresponding to the structures in the historical image data may bedetermined, by a doctor manually and/or by a computing deviceautomatically. The training sample set may be stored in the storagedevice of the imaging system 100 and retrieved by the processing device120 from the storage device. Alternatively, the training sample set maybe generated by the processing device 120. For example, the processingdevice 120 may pre-process historical image data representing structuresof a plurality of subject with center points of the structures labeled.Merely by way of example, the historical image data may be resample togenerate resampled image data having a target image resolution.Optionally, the processing device 120B may further normalize theresampled image data. As another example, the processing device 120 mayextract one or more image crops from the resampled image data, andnormalize each of the image crop(s).

In 820, the preliminary classification model may be trained using thetraining sample set to obtain a trained classification model.

In some embodiments, the preliminary classification model to be trainedmay include one or more model parameters. Exemplary model parameters mayinclude the number (or count) of layers, the number (or count) of nodes,or the like, or any combination thereof. Before training, thepreliminary classification model may have one or more initial parametervalues of the model parameter(s). In the training of the preliminarymodel, the value(s) of the model parameter(s) of the preliminary modelmay be updated.

In some embodiments, the training of the preliminary classificationmodel may include one or more iterations to iteratively update the modelparameters of the preliminary model until a termination condition issatisfied in a certain iteration. Exemplary termination conditions maybe that the value of an objective function (i.e., loss function)obtained in the certain iteration is less than a threshold value, that acertain count of iterations have been performed, that the objectivefunction converges such that the difference of the values of theobjective function obtained in a previous iteration and the currentiteration is within a threshold value, etc. Exemplary objectivefunctions may include a focal loss function, a log loss function, across-entropy loss function, a Dice loss function, etc.

In some embodiments, a focal loss function may be used as the objectivefunction. A loss value may be determined by comparing an output of theclassification model with a target classification of a structurecorresponding to historical image data input into the classificationmodel (i.e., the target classification and the historical image data mayconstitute a training sample pair). In some embodiments, the preliminaryclassification model may be trained in a plurality of iterations. Duringthe plurality of iterations, an optimal solution of the focal lossfunction may be obtained.

In some embodiments, each of the plurality of candidate classificationsmay correspond to a weight in the focal loss function. At least oneweight of the focal loss function each of which corresponds to one ofthe plurality of candidate classifications being different from weightsof the focal loss function corresponding to the remainder of theplurality of candidate classifications. For example, a smaller weightmay be assigned to a benign pulmonary nodule, and a larger weight may beassigned to a malignant pulmonary nodule. By adjusting the weightscorresponding to the plurality of candidate classifications, negativeeffects induced by imbalance of samples corresponding to differentcandidate classifications on the classification model may be reduced,and a more robust classification model may be obtained. In the presentdisclosure, the terms “classification model” and “trained classificationmodel” may be used interchangeably. The use of the focal loss functionas the objective function may improve the performance of theclassification model.

Merely for illustration purposes, the historical image data may be inputinto the preliminary classification model in a particular batch size(e.g., a batch size of 10, 24, 48, 100, etc.) to train the preliminaryclassification model. In some embodiments, in the training process ofthe preliminary classification model, all the historical image data inthe particular batch size may be input into the preliminaryclassification model to train the preliminary classification model.

In some embodiments, the historical image data may be input into thepreliminary classification backbone network, the preliminarysegmentation network, and the preliminary density networksimultaneously. In some embodiments, the historical image data may beinput into the preliminary classification backbone network, thepreliminary segmentation network, and the preliminary density networkindividually or in sequence. For example, the preliminary segmentationnetwork and the preliminary density network may be trained in advance,and the historical image data may be input into the preliminaryclassification backbone network to train the preliminary classificationmodel. In some embodiments, the preliminary classification model may betrained in a plurality of iterations. The trained classification modelmay be saved as a file in a storage device (e.g., the storage device130).

In some embodiments, before the generation of the training sample setbased on the historical image data, the structures, and the targetclassifications corresponding to the structures in the historical imagedata, a pre-processing operation (e.g., resampling, normalization,denoising, etc.) may be performed on the historical image data.

In 830, image data representing a structure of a subject may beobtained. The image data may be obtained by the obtaining module 410.

In 840, a plurality of candidate classifications of the structure andtheir respective probabilities may be determined by inputting the imagedata into the trained classification model.

In 850, a target classification of the structure may be determined basedon at least a part of the probabilities of the plurality of candidateclassifications. In some embodiments, the operations 830 through 850 maybe the same as or similarly to the operations 510 through 530 of theprocess 500 as illustrated in FIG. 5.

FIG. 9 includes a schematic diagram illustrating the determination of atarget classification of a structure of a subject in an image accordingto some embodiments of the present disclosure. As shown in FIG. 9, thedetermination of the target classification of the structure may includetwo phases including a model training phase and a classification phases.A classification model (e.g., a neural network model) for classificationmay be generated during the model training phase. The classificationmodel may include a plurality of parameters. The plurality of parametersmay be determined through a training process. The parameters may beloaded in the classification model to determine a target classificationof a structure in an image input by a user. In some embodiments, amachine learning platform 910 may train a classification model 920 usinga training sample set. The training sample set may include historicalimage data representing structures of a plurality of subject generatedusing the imaging device 110, whereins the structures may be labeled inthe historical image data and target classifications corresponding tothe structures in the historical image data may be determined, by adoctor manually and/or by a computing device automatically. In someembodiments, the structures may be labeled by marking center points ofthe structures in the training sample set. The process for training theclassification model may enable the model to learn a general rule thatmaps inputs to corresponding outputs. Exemplary algorithms that themachine learning platform may use to train the model may include agradient boosting decision tree (GBDT) algorithm, a decision treealgorithm, a Random Forest algorithm, a logistic regression algorithm, asupport vector machine (SVM) algorithm, a Naive Bayesian algorithm, anAdaBoost algorithm, a K-a nearest neighbor (KNN) algorithm, a MarkovChains algorithm, or the like, or any combination thereof.

In some embodiments, the classification model may be trained in aplurality of iterations. In some embodiments, the training process mayterminate when the focal loss function reaches a preset threshold, and atrained classification model 930 may be obtained. The trainedclassification model 930 may be saved as a file in a storage device ofthe imaging system 100. In some embodiments, the trained classificationmodel 930 may be incorporated into a classification algorithm 940 tofacilitate the target classification determination of the structure. Aclient terminal 950 may receive image data representing a structure tobe classified, and determine a target classification of the structure tobe classified using the classification algorithm 940. The targetclassification of the structure may be feed back to the client terminal950, which may be used for medical diagnosis of the structure (e.g., ina form of a diagnostic report).

Technical solutions disclosed in the above embodiments of presentdisclosure may provide an operation of training the classification modelusing the historical image data and the target classificationscorresponding to the historical image data. The training sample set maybe generated based on the historical image data and the targetclassifications corresponding to the historical image data. Theclassification model may be trained using the training sample set and afocal loss function. In some embodiments, each of the plurality ofcandidate classifications may correspond to a weight in the focal lossfunction. By adjusting the weights corresponding to the plurality ofcandidate classifications, negative effects induced by imbalance ofsamples corresponding to different candidate classifications on theclassification model may be reduced, and a more robust classificationmodel may be obtained. The use of the focal loss function as theobjective function may improve the performance of the classificationmodel.

FIG. 10 is a schematic diagram of an exemplary computing apparatusaccording to some embodiments of the present disclosure. The computingapparatus 1012 may facilitate the implementation of the processes oroperations provided in the present disclosure. The computing apparatus1012 illustrated in FIG. 10 is merely an example, but not intended tolimit the scope of the present disclosure.

As shown in FIG. 10, the computing apparatus 1012 may be implemented bya computing device of general purposes. The computing apparatus 1012 mayinclude but are not limited to one or more processors 1016, a systemmemory 1028, and a bus 1018 that connects elements or components of thecomputing apparatus 1012, such as the system memory 1028, the one ormore processors 1016, etc.

The bus 1018 may represent one or more of several types of busstructures, including a memory bus, a memory controller, peripheral bus,an accelerated graphics port, the one or more processors 1016, or alocal bus using any of a variety of bus structures. For example, the busstructures may include but not limited to, an Industry StandardArchitecture (ISA) bus, a Micro Channel Architecture (MAC) bus, anEnhanced ISA Bus, a Video Electronics Standards Association (VESA) localbus, a peripheral component interconnects (PCI) bus, etc.

The computing apparatus 1012 may include a variety of computer readablemedia. The computer readable media may be any available media includingvolatile or non-volatile media, removable or non-removable media, etc.,that may be accessible by the computing apparatus 1012.

The system memory 1028 may include computer readable media in a form ofvolatile memory, for example, a random access memory (RAM) 1030 and/or aread-only memory (ROM) 1032. The computing apparatus 1012 may furtherinclude other removable/non-removable or volatile/non-volatile computersystem storage media. Merely by ways of example, a storage device 1034may be non-removable, non-volatile magnetic media (not shown in thefigure, commonly referred to as a “hard disk drive”) for reading andwriting. Although not shown in FIG. 10, a disk drive for reading andwriting to a removable non-volatile disk (such as a “floppy disk”) and aremovable non-volatile disk (such as a CD-ROM, a DVD-ROM, or otheroptical media) may be provided. In these cases, each drive may becoupled to the bus 1018 via one or more data medium ports. The systemmemory 1028 may include at least one program product having a set (e.g.,at least one) of program modules configured to implement the functionsprovided in the above embodiments of the present disclosure.

A program/utility tool 1040 having a set (at least one) of programmodules 1042, which may be stored, for example, in the memory 1028. Theprogram modules 1042 may include but not limited to, an operatingsystem, one or more applications, other program modules, or programdata. Each or a combination of one or more of the above listed programmodules may have a network environment implementation. The programmodule 1042 may perform the functions and/or methods provided in thedescribed embodiments of the present disclosure.

The computing apparatus 1012 may also be in communication with one ormore external devices 1014 (e.g., a keyboard, a pointing device, adisplay 1024, etc.), one or more devices that enable a user to interactwith the computing apparatus 1012, and/or any devices (e.g., a networkcard, a modem, etc.) that enable the computing apparatus 1012 tocommunicate with one or more other computing devices. The communicationmay be realized via an input/output (I/O) interface 1022. Also, thecomputing apparatus 1012 may also communicate with one or more networks(e.g., a local area network (LAN), a wide area network (WAN), and/or apublic network, such as the Internet) through a network adapter 1020. Asshown in the figure, the network adapter 1020 may communicate with othermodules of computing apparatus 1012 via the bus 1018. It should beunderstood that, other hardware and/or software modules may be utilizedin combination with the computing apparatus 1012, including but notlimited to microcode, device drivers, redundant processing units,external disk drive arrays, RAID systems, Tape drives, or data backupstorage systems.

The one or more processors 1016 may implement, by running a programstored in the system memory 1028, various functional applications and/ordata processing, for example, a method of classification determinationof a structure of a subject in an image provided in some embodiments ofthe present disclosure. According to an aspect of the presentdisclosure, the method may include obtaining image data representing astructure of a subject. The method may also include determining aplurality of candidate classifications of the structure and theirrespective probabilities by inputting the image data into aclassification model, wherein the classification model is trainedaccording to a focal loss function, at least one weight of the focalloss function each of which corresponds to one of the plurality ofcandidate classifications being different from weights of the focal lossfunction corresponding to the remainder of the plurality of candidateclassifications. The method may further include determining a targetclassification of the structure based on the probabilities of theplurality of candidate classifications.

Those skilled in the art may understand that the one or more processors1016 may also implement technical solutions of the exposure processcontrol method provided by any embodiments of the present disclosure.

The present disclosure may further provide a computer readable storagemedium storing computer programs. When the computer programs areexecuted by a processor, operations of classification determination of astructure of a subject in an image provided in the present disclosuremay be implemented. According to a first aspect of the presentdisclosure, the operations may include obtaining image data representinga structure of a subject. The operations may also include determining aplurality of candidate classifications of the structure and theirrespective probabilities by inputting the image data into aclassification model, wherein the classification model is trainedaccording to a focal loss function, at least one weight of the focalloss function each of which corresponds to one of the plurality ofcandidate classifications being different from weights of the focal lossfunction corresponding to the remainder of the plurality of candidateclassifications. The operations may further include determining a targetclassification of the structure based on the probabilities of theplurality of candidate classifications.

It should be noted that the computer programs stored in the computerreadable storage medium may not limited to the methods or operationsprovided above, other methods or operations related to the automatedpositioning of the subject may also be provided.

Having thus described the basic concepts, it may be rather apparent tothose skilled in the art after reading this detailed disclosure that theforegoing detailed disclosure is intended to be presented by way ofexample only and is not limiting. Various alterations, improvements, andmodifications may occur and are intended to those skilled in the art,though not expressly stated herein. These alterations, improvements, andmodifications are intended to be suggested by this disclosure, and arewithin the spirit and scope of the exemplary embodiments of thisdisclosure.

Moreover, certain terminology has been used to describe embodiments ofthe present disclosure. For example, the terms “one embodiment,” “anembodiment,” and/or “some embodiments” mean that a particular feature,structure or feature described in connection with the embodiment isincluded in at least one embodiment of the present disclosure.Therefore, it is emphasized and should be appreciated that two or morereferences to “an embodiment” or “one embodiment” or “an alternativeembodiment” in various portions of this specification are notnecessarily all referring to the same embodiment. Furthermore, theparticular features, structures or features may be combined as suitablein one or more embodiments of the present disclosure.

Further, it will be appreciated by one skilled in the art, aspects ofthe present disclosure may be illustrated and described herein in any ofa number of patentable classes or context including any new and usefulprocess, machine, manufacture, or composition of matter, or any new anduseful improvement thereof. Accordingly, aspects of the presentdisclosure may be implemented entirely hardware, entirely software(including firmware, resident software, micro-code, etc.) or combiningsoftware and hardware implementation that may all generally be referredto herein as a “unit,” “module,” or “system.” Furthermore, aspects ofthe present disclosure may take the form of a computer program productembodied in one or more computer readable media having computer readableprogram code embodied thereon.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including electro-magnetic, optical, or thelike, or any suitable combination thereof. A computer readable signalmedium may be any computer readable medium that is not a computerreadable storage medium and that may communicate, propagate, ortransport a program for use by or in connection with an instructionexecution system, apparatus, or device. Program code embodied on acomputer readable signal medium may be transmitted using any appropriatemedium, including wireless, wireline, optical fiber cable, RF, or thelike, or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of thepresent disclosure may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C #, VB.NET, Python or the like, conventional procedural programming languages,such as the “C” programming language, Visual Basic, Fortran 2103, Perl,COBOL 2102, PHP, ABAP, dynamic programming languages such as Python,Ruby and Groovy, or other programming languages. The program code mayexecute entirely on the user's computer, partly on the user's computer,as a stand-alone software package, partly on the user's computer andpartly on a remote computer or entirely on the remote computer orserver. In the latter scenario, the remote computer may be connected tothe user's computer through any type of network, including a local areanetwork (LAN) or a wide area network (WAN), or the connection may bemade to an external computer (for example, through the Internet using anInternet Service Provider) or in a cloud computing environment oroffered as a service such as a Software as a Service (SaaS).

Furthermore, the recited order of processing elements or sequences, orthe use of numbers, letters, or other designations therefore, is notintended to limit the claimed processes and methods to any order exceptas may be specified in the claims. Although the above disclosurediscusses through various examples what is currently considered to be avariety of useful embodiments of the disclosure, it is to be understoodthat such detail is solely for that purpose, and that the appendedclaims are not limited to the disclosed embodiments, but, on thecontrary, are intended to cover modifications and equivalentarrangements that are within the spirit and scope of the disclosedembodiments. For example, although the implementation of variouscomponents described above may be embodied in a hardware device, it mayalso be implemented as a software only solution, for example, aninstallation on an existing server or mobile device.

Similarly, it should be appreciated that in the foregoing description ofembodiments of the present disclosure, various features are sometimesgrouped together in a single embodiment, figure, or description thereoffor the purpose of streamlining the disclosure aiding in theunderstanding of one or more of the various inventive embodiments. Thismethod of disclosure, however, is not to be interpreted as reflecting anintention that the claimed subject matter requires more features thanare expressly recited in each claim. Rather, inventive embodiments liein less than all features of a single foregoing disclosed embodiment.

In some embodiments, the numbers expressing quantities or propertiesused to describe and claim certain embodiments of the application are tobe understood as being modified in some instances by the term “about,”“approximate,” or “substantially.” For example, “about,” “approximate,”or “substantially” may indicate ±20% variation of the value itdescribes, unless otherwise stated. Accordingly, in some embodiments,the numerical parameters set forth in the written description andattached claims are approximations that may vary depending upon thedesired properties sought to be obtained by a particular embodiment. Insome embodiments, the numerical parameters should be construed in lightof the number of reported significant digits and by applying ordinaryrounding techniques. Notwithstanding that the numerical ranges andparameters setting forth the broad scope of some embodiments of theapplication are approximations, the numerical values set forth in thespecific examples are reported as precisely as practicable.

Each of the patents, patent applications, publications of patentapplications, and other material, such as articles, books,specifications, publications, documents, things, and/or the like,referenced herein is hereby incorporated herein by this reference in itsentirety for all purposes, excepting any prosecution file historyassociated with same, any of same that is inconsistent with or inconflict with the present document, or any of same that may have alimiting affect as to the broadest scope of the claims now or laterassociated with the present document. By way of example, should there beany inconsistency or conflict between the description, definition,and/or the use of a term associated with any of the incorporatedmaterial and that associated with the present document, the description,definition, and/or the use of the term in the present document shallprevail.

In closing, it is to be understood that the embodiments of theapplication disclosed herein are illustrative of the principles of theembodiments of the application. Other modifications that may be employedmay be within the scope of the application. Thus, by way of example, butnot of limitation, alternative configurations of the embodiments of theapplication may be utilized in accordance with the teachings herein.Accordingly, embodiments of the present application are not limited tothat precisely as shown and described.

What is claimed is:
 1. A system, comprising: at least one storage mediumincluding a set of instructions; and at least one processor configuredto communicate with the at least one storage medium, wherein whenexecuting the set of instructions, the system is directed to performoperations including: obtaining image data representing a structure of asubject; obtaining a backbone feature, a segmentation feature, and adensity feature of the structure by inputting the image data into aclassification model, wherein the classification model includes abackbone network for determining the backbone feature of the structure,a segmentation network for determining the segmentation feature of thestructure, and a density classification network for determining thedensity feature of the structure; converting the backbone feature, thesegmentation feature, and the density feature into a one-dimensionalbackbone feature vector, a one-dimensional segmentation feature vector,and a one-dimensional density feature vector, respectively; determininga one-dimensional identification feature vector by splicing theone-dimensional backbone feature vector, the one-dimensionalsegmentation feature vector, and the one-dimensional density featurevector; and determining a plurality of candidate classifications of thestructure and their respective probabilities based on theone-dimensional identification feature vector; and determining a targetclassification of the structure based on at least a part of theprobabilities of the plurality of candidate classifications.
 2. Thesystem of claim 1, wherein the backbone network includes a plurality ofdown-sampling layers, each down-sampling layer including a convolutionlayer, a batch normalization layer, and a rectified linear unit layer.3. The system of claim 1, wherein the classification model is trainedaccording to a focal loss function, at least one weight of the focalloss function each of which corresponds to one of the plurality ofcandidate classifications being different from weights of the focal lossfunction corresponding to the remainder of the plurality of candidateclassifications.
 4. The system of claim 1, the obtaining image datarepresenting a structure of a subject including: obtaining originalimage data including a representation of the structure of the subject;and determining the image data by preprocessing the original image data.5. The system of claim 4, the determining the image data bypreprocessing the original image data including: generating a resampledimage by resampling the original image data according to a resamplingresolution; segmenting the resampled image into image crops according toa center of the structure; and determining the image data by normalizingthe image crops according to a normalizing function.
 6. A methodimplemented on a computing device having a processor and acomputer-readable storage device, the method comprising: obtaining, bythe computing device, image data representing a structure of a subject;obtaining, by the computing device, a backbone feature, a segmentationfeature, and a density feature of the structure by inputting the imagedata into a classification model, wherein the classification modelincludes a backbone network for determining the backbone feature of thestructure, a segmentation network for determining the segmentationfeature of the structure, and a density classification network fordetermining the density feature of the structure; converting, by thecomputing device, the backbone feature, the segmentation feature, andthe density feature into a one-dimensional backbone feature vector, aone-dimensional segmentation feature vector, and a one-dimensionaldensity feature vector, respectively; determining, by the computingdevice, a one-dimensional identification feature vector by splicing theone-dimensional backbone feature vector, the one-dimensionalsegmentation feature vector, and the one-dimensional density featurevector; and determining, by the computing device, a plurality ofcandidate classifications of the structure and their respectiveprobabilities based on the one-dimensional identification featurevector; and determining, by the computing device, a targetclassification of the structure based on at least a part of theprobabilities of the plurality of candidate classifications.
 7. Themethod of claim 6, wherein the backbone network includes a plurality ofdown-sampling layers, each down-sampling layer including a convolutionlayer, a batch normalization layer, and a rectified linear unit layer.8. The method of claim 6, wherein the classification model is trainedaccording to a focal loss function, at least one weight of the focalloss function each of which corresponds to one of the plurality ofcandidate classifications being different from weights of the focal lossfunction corresponding to the remainder of the plurality of candidateclassifications.
 9. The method of claim 6, the obtaining image datarepresenting a structure of a subject including: obtaining originalimage data including a representation of the structure of the subject;and determining the image data by preprocessing the original image data.10. The method of claim 9, the determining the image data bypreprocessing the original image data including: generating a resampledimage by resampling the original image data according to a resamplingresolution; segmenting the resampled image into image crops according toa center of the structure; and determining the image data by normalizingthe image crops according to a normalizing function.
 11. Anon-transitory computer readable medium, comprising at least one set ofinstructions, wherein when executed by at least one processor of acomputing device, the at least one set of instructions causes thecomputing device to perform a method, the method comprising: obtainingimage data representing a structure of a subject; obtaining a backbonefeature, a segmentation feature, and a density feature of the structureby inputting the image data into a classification model, wherein theclassification model includes a backbone network for determining thebackbone feature of the structure, a segmentation network fordetermining the segmentation feature of the structure, and a densityclassification network for determining the density feature of thestructure; converting the backbone feature, the segmentation feature,and the density feature into a one-dimensional backbone feature vector,a one-dimensional segmentation feature vector, and a one-dimensionaldensity feature vector, respectively; determining a one-dimensionalidentification feature vector by splicing the one-dimensional backbonefeature vector, the one-dimensional segmentation feature vector, and theone-dimensional density feature vector; and determining a plurality ofcandidate classifications of the structure and their respectiveprobabilities based on the one-dimensional identification featurevector; and determining a target classification of the structure basedon at least a part of the probabilities of the plurality of candidateclassifications.
 12. The non-transitory computer readable medium ofclaim 11, wherein the backbone network includes a plurality ofdown-sampling layers, each down-sampling layer including a convolutionlayer, a batch normalization layer, and a rectified linear unit layer.13. The non-transitory computer readable medium of claim 11, wherein theclassification model is trained according to a focal loss function, atleast one weight of the focal loss function each of which corresponds toone of the plurality of candidate classifications being different fromweights of the focal loss function corresponding to the remainder of theplurality of candidate classifications.
 14. The non-transitory computerreadable medium of claim 11, the obtaining image data representing astructure of a subject including: obtaining original image dataincluding a representation of the structure of the subject; anddetermining the image data by preprocessing the original image data. 15.The non-transitory computer readable medium of claim 14, the determiningthe image data by preprocessing the original image data including:generating a resampled image by resampling the original image dataaccording to a resampling resolution; segmenting the resampled imageinto image crops according to a center of the structure; and determiningthe image data by normalizing the image crops according to a normalizingfunction.